Use Github Action to deploy to AWS

Trong Minh Duc Hoang | Feb 7, 2023

Overview

In this blog, we will use this infrastructure to do a step-by-step demo.

Prerequisite

IAM credentials (AWS access key ID, AWS Secret access key, region name)

Github repositories

Docker

NodeJs (yarn)

Deploy static website to S3 with Github Action

1. Create AWS S3 bucket and enable ACLs

Enable public access

2. Create workflow for Github action

In this demo, we are deploying an ReactJs app. Let’s explain some steps to build this app:

Run yarn install to install all dependencies.

Run yarn build to build the app. The content of output folder will be pushed to S3. In my case, the output folder is dist.

Create a workflow file inside the repository .github/workflows/deploy.yml

name: Production Build
on:
  pull_request:
  push:
    branches:
      - main # the branch and action we want to trigger this workflow
jobs:
  build:
    runs-on: ubuntu-latest # We use Uubuntu to run this workflow on
    
    strategy:
      matrix:
        node-version: [ 16.x ] # Set NodeJs version here
    steps:
    - uses: actions/checkout@v3 # Check out the repo
    - name: Use Node.js ${{ matrix.node-version }}
      uses: actions/setup-node@v3 # Install NodeJs in this Ubuntu
      with:
        node-version: ${{ matrix.node-version }}
    - name: Yarn Install # Run command to install all dependencies
      run: |
        yarn install
    - name: Production Build # Build the file, the output folder in this case is dist
      run: |
        yarn build
    - name: Deploy to S3
      uses: jakejarvis/s3-sync-action@master
      with:
        args: --acl public-read --delete
      env: # All the keys we need from AWS will be passed into this action
        AWS_S3_BUCKET: ${{ secrets.AWS_PRODUCTION_BUCKET_NAME }}
        AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
        AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
        AWS_REGION: ${{ secrets.AWS_REGION }}
        SOURCE_DIR: "dist" # The output folder of build step above (it depends on your app)

3. Set env secrets

As you can see, we use four secret keys in above workflow. We need to define it in our repository’s setting and give it a value.

4. Push code to Github and enjoy the automation

Github Action will automatically trigger this workflow when we push this workflow definition along with the code to the repo

To learn more about Github Action, you could go to this link to find an action that fits your need https://github.com/marketplace?type=actions&query=s3+

5. Access S3 from Cloudfront

Create a new distribution on Cloudfron to serve files from the S3 bucket. Now we can access our app publicly through Cloudfront.

6. Do some tricks after deployed

There is a problem each time we deploy the app, which is cached files on Cloudfront, so we won’t see new changes on the website because the cached files are valid within 24 hours. That is why we want to create invalidation after each deployment. The below code shows a function to do it.

We also want to know when is the deployment finished. In this case, we will use SNS to send an email to our inbox to inform the deployment is finished and ready for production.

a. Create a role for Lambda

Create a role for this Lambda that are enabled to publish message to SNS and create Cloudfront invalidation

b. Create a SNS topic and subcribe our email to it.

c. Create a Lambda

Create a Lambda NodeJs 18 and assign above role to it.

Fill out YOUR_REGION, YOUR_TOPIC_ARN, and YOUR_DISTRIBUTION_ID.

import { SNSClient, PublishCommand } from "@aws-sdk/client-sns";
import { CloudFrontClient, CreateInvalidationCommand } from "@aws-sdk/client-cloudfront";

const YOUR_REGION = 'YOUR_REGION';
const YOUR_TOPIC_ARN = 'YOUR_TOPIC_ARN';
const YOUR_DISTRIBUTION_ID = 'YOUR_DISTRIBUTION_ID';


const sendNotiToEmail = async () => {
    const client = new SNSClient({ region: YOUR_REGION });
    const params = {
        Message: `New version deployed at ${new Date(Date.now()).toLocaleString("en-US", { timeZone: "America/Chicago" })}`,
        TopicArn: YOUR_TOPIC_ARN
    };
    const command = new PublishCommand(params);
    const response = await client.send(command);
    return response;
}

const clearCloudFrontCache = async () => {
    const client = new CloudFrontClient({ region: YOUR_REGION });
    const command = new CreateInvalidationCommand({
        DistributionId: YOUR_DISTRIBUTION_ID,
        InvalidationBatch: {
            CallerReference: new Date().getTime().toString(),
            Paths: {
                Quantity: 1,
                Items: ['/*'] // Clear all cached files
            }
        }
    });
    return client.send(command);
}

export const handler = async (event) => {

    await sendNotiToEmail();
    await clearCloudFrontCache();
    return {
        statusCode: 200,
        body: "Done"
    }
};

d. Create a trigger from S3 bucket

Create a trigger from S3 bucket when index.html has an update to run this lambda function after deployed

Deploy backend NextJs application to ECS

1. Create security groups

a. Create a security group for load balancer

Allow all access from the internet (MyLoadBalancerSg)

b. Create a security group for ECS container

Only allow access from MyLoadBalancerSg

2. Create a repository in ECR

3. Build & push first image

We will dockerize our NextJs application and push it to ECR repository.

Create a file Dockerfile inside our code base

# Use an official Node.js image as the base image
FROM node:16-alpine

# Set the working directory in the image to /app
WORKDIR /app

# Copy the rest of the application code to the image
COPY . .

# Install the application dependencies
RUN yarn install
# Build nextjs application, output folder is .next
RUN yarn build

# Specify the command to start the Next.js application
CMD ["npm", "run", "start"]
EXPOSE 3000

In our ECR repository, click on View push commands then follow these steps to push our first image to this repo

4. Create task definition

Go to ECS, click on Task Definition, and click on Create Task Definition

Create a task definition using the URI from above ECR (e.g: http://856210122328.drecr.us-east-1.amazonaws.com/mywebiste-be:latest)

Map the port we want to expose from containers. In this case, NextJs’ port is 3000

Pass env variables as your wish to containers

5. Create ECS cluster and service using Cloudformation

We will use this Cloudformation template to create the rest of the service. We can still do it manually on the UI but when we create an ECS service inside the ECS cluster, there is a glitch that makes the Load Balancer section disappear so we cannot create a load balancer along with our ECS service. By doing it, Cloudformation will make sure we always have a load balancer when creating a ECS service.

Create a yaml file for this template (e.g: myecsstack.yml)

Replace all the parameters in the template

AWSTemplateFormatVersion: 2010-09-09
Description: The template used to create an ECS Service from the ECS Console.
Parameters:
    # (REPLACE THIS) Enter the security group IDs for the load balancer and the container that were created in the previous step.
    LoadBalancerSecurityGroup:
        Type: CommaDelimitedList
        Default: sg-04e9e5c992ae5d43c
    ContainerSecurityGroup:
        Type: CommaDelimitedList
        Default: sg-011d187169c541958
    # (REPLACE THIS) Enter the subnet IDs for the VPC where the load balancer and the container will be created.
    SubnetIDs:
        Type: CommaDelimitedList
        Default: >-
            subnet-0f1cffba22f6b4742,subnet-01a6bb4ca617d76e6,subnet-066dd3cb1343aa92d,subnet-06431437c9d252d83,subnet-05ece4ddfab51758e,subnet-0d719136bbfff4870
    VpcID:
        Type: String
        Default: vpc-04b4663f76ce16dd8
		# (REPLACE THIS) Create a new role with `AmazonECSTaskExecutionRolePolicy` and enter its arn here
    TaskRole:
        Type: String
        Default: arn:aws:iam::856210122328:role/ecsTaskExecutionRole
		# (REPLACE THIS) Enter the task definition's arn that we created from above step
		TaskDefinitionArn:
        Type: String
        Default: arn:aws:ecs:us-east-1:856210122328:task-definition/mywebsite:1
    # Choose whatever you want for below parameters
    ECSClusterName:
        Type: String
        Default: mywebsite-cluster
    LoadBalancerName:
        Type: String
        Default: MyLoadBalancerECS
    ServiceName:
        Type: String
        Default: mywebsite-service
    ContainerName:
        Type: String
        Default: mywebsite
    ContainerPort:
        Type: Number
        Default: 3000 # matching with the exposed port from docker
    TargetGroupName:
        Type: String
        Default: MyTargetGroupECS
Resources:
    ECSCluster:
        Type: 'AWS::ECS::Cluster'
        Properties:
            ClusterName: !Ref ECSClusterName
    ECSService:
        Type: 'AWS::ECS::Service'
        Properties:
            Cluster: !Ref ECSClusterName
            CapacityProviderStrategy:
                - CapacityProvider: FARGATE
                  Base: 0
                  Weight: 1
            TaskDefinition: !Ref TaskDefinitionArn
            ServiceName: !Ref ServiceName
            SchedulingStrategy: REPLICA
            DesiredCount: 1
            LoadBalancers:
                - ContainerName: !Ref ContainerName
                  ContainerPort: !Ref ContainerPort
                  LoadBalancerName: !Ref 'AWS::NoValue'
                  TargetGroupArn: !Ref TargetGroup
            NetworkConfiguration:
                AwsvpcConfiguration:
                    AssignPublicIp: ENABLED
                    SecurityGroups: !Ref ContainerSecurityGroup
                    Subnets: !Ref SubnetIDs
            PlatformVersion: LATEST
            DeploymentConfiguration:
                MaximumPercent: 200
                MinimumHealthyPercent: 100
                DeploymentCircuitBreaker:
                    Enable: true
                    Rollback: true
            DeploymentController:
                Type: ECS
            ServiceConnectConfiguration:
                Enabled: false
            Tags:
                - Key: 'ecs:service:stackId'
                  Value: !Ref 'AWS::StackId'
            EnableECSManagedTags: true
        DependsOn:
            - Listener
    LoadBalancer:
        Type: 'AWS::ElasticLoadBalancingV2::LoadBalancer'
        Properties:
            Type: application
            Name: !Ref LoadBalancerName
            SecurityGroups: !Ref LoadBalancerSecurityGroup
            Subnets: !Ref SubnetIDs
    TargetGroup:
        Type: 'AWS::ElasticLoadBalancingV2::TargetGroup'
        Properties:
            HealthCheckPath: /
            Name: !Ref TargetGroupName
            Port: !Ref ContainerPort
            Protocol: HTTP
            TargetType: ip
            HealthCheckProtocol: HTTP
            VpcId: !Ref VpcID
    Listener:
        Type: 'AWS::ElasticLoadBalancingV2::Listener'
        Properties:
            DefaultActions:
                - Type: forward
                  TargetGroupArn: !Ref TargetGroup
            LoadBalancerArn: !Ref LoadBalancer
            Port: 80
            Protocol: HTTP

After done with the template, go to Cloudformation and upload this file to create these services.

Check if the load balancer is working as expected

6. Create workflow for Github Action

There is an official workflow to deploy AWS ECS on Github. You could go to your repository Click on Actions \ New workflow , and search for ecs

But, in this guideline, we will customize the official workflow a little bit.

We will download task definition and export it to a json file instead of providing a preset task definition json in our repo.

Add a last step to send email to our inbox after everything is done Notify SNS using the same SNS topic that we created from previous section so we know when everything is done.


name: Deploy to Amazon ECS

on:
  push:
    branches: [ "main" ]

env:
  AWS_REGION: ${{ secrets.AWS_REGION }}             # set this to your preferred AWS region, e.g. us-west-1
  ECR_REPOSITORY: ${{ secrets.ECR_REPOSITORY }}   # set this to your Amazon ECR repository name
  ECS_SERVICE: ${{secrets.ECS_SERVICE}}                 # set this to your Amazon ECS service name
  ECS_CLUSTER: ${{secrets.ECS_CLUSTER}}                 # set this to your Amazon ECS cluster name
  CONTAINER_NAME: ${{secrets.CONTAINER_NAME}}          # set this to the name of the container in the
  ECS_TASK_DEFINITION_NAME: ${{secrets.ECS_TASK_DEFINITION_NAME}}
permissions:
  contents: read

jobs:
  deploy:
    name: Deploy
    runs-on: ubuntu-latest
    environment: production

    steps:
    - name: Checkout
      uses: actions/checkout@v3

    - name: Configure AWS credentials
      uses: aws-actions/configure-aws-credentials@v1
      with:
        aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
        aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
        aws-region: ${{ env.AWS_REGION }}

    - name: Login to Amazon ECR
      id: login-ecr
      uses: aws-actions/amazon-ecr-login@v1

    - name: Download task definition
      run: |
        aws ecs describe-task-definition --task-definition "${{secrets.ECS_TASK_DEFINITION_NAME}}" --query taskDefinition > task-definition.json

    - name: Build, tag, and push image to Amazon ECR
      id: build-image
      env:
        ECR_REGISTRY: ${{ steps.login-ecr.outputs.registry }}
        IMAGE_TAG: ${{ github.sha }}
      run: |
        # Build a docker container and
        # push it to ECR so that it can
        # be deployed to ECS.
        docker build -t $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG .
        docker push $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG
        echo "image=$ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG" >> $GITHUB_OUTPUT

    - name: Fill in the new image ID in the Amazon ECS task definition
      id: task-def
      uses: aws-actions/amazon-ecs-render-task-definition@v1
      with:
        task-definition: task-definition.json
        container-name: ${{ env.CONTAINER_NAME }}
        image: ${{ steps.build-image.outputs.image }}

    - name: Deploy Amazon ECS task definition
      uses: aws-actions/amazon-ecs-deploy-task-definition@v1
      with:
        task-definition: ${{ steps.task-def.outputs.task-definition }}
        service: ${{ env.ECS_SERVICE }}
        cluster: ${{ env.ECS_CLUSTER }}
        wait-for-service-stability: true
    
    - name: Notify SNS
      run: |
        aws sns publish --topic-arn "${{secrets.SNS_ARN}}" --message "New Backend Deployment at $(date +%c)"

a. Set env variables that we need for this workflow

AWS_REGION

ECR_REPOSITORY

ECS_SERVICE

ECS_CLUSTER

CONTAINER_NAME

ECS_TASK_DEFINITION_NAME

AWS_ACCESS_KEY_ID

AWS_SECRET_ACCESS_KEY

SNS_ARN

b. Create the workflow file in the source code and push it to Github

Finally, a new task definition will be deployed in the ECS service and replace the previous version. Its private IP will be automatically registered to the target group that we created along with the load balancer and ECS service.

There is something you can do to optimize this workflow:

Reduce the size of the docker image

Clean up the unused images in ECR because it will cost us a lot